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Greedy  Algorithms  in  Banach  Spaces1 


V.N.Temlyakov 

Department  of  Mathematics,  University  of  South  Carolina,  Columbia,  SC  29208 

Abstract.  We  study  efficiency  of  approximation  and  convergence  of  two  greedy 
type  algorithms  in  uniformly  smooth  Banach  spaces.  The  Weak  Chebyshev  Greedy 
Algorithm  (WCGA)  is  defined  for  an  arbitrary  dictionary  T>  and  provides  nonlinear 
m- term  approximation  with  regard  to  V.  This  algorithm  is  defined  inductively  with 
the  m-th  step  consisting  of  two  basic  substeps:  1)  selection  of  an  m-th  element 
from  T>  and  2)  constructing  an  m- term  approximant  G^.  We  include  the  name  of 
Chebyshev  in  the  name  of  this  algorithm  because  at  the  substep  2)  the  approximant 
G^  is  chosen  as  the  best  approximant  from  span(<p^, . . . ,  <p^).  The  term  Weak 
Greedy  Algorithm  indicates  that  at  each  substep  1)  we  choose  as  an  element  of 
V  that  satisfies  some  condition  which  is  ”tm-times  weaker”  than  the  condition  for 
to  be  optimal  (tm  =  1).  We  got  error  estimates  for  Banach  spaces  with  modulus 
of  smoothness  p{u)  <  yw9,  1  <  q  <  oo.  We  proved  that  for  any  /  from  the  closure 
of  the  convex  hull  of  DUO  the  error  of  m-term  approximation  by  WCGA  is  of  order 
(l  +  flj>  +  --  -  +  tm)-1/p>  l/f>+l/g  =  l.  Similar  results  are  obtained  for  Weak  Relaxed 
Greedy  Algorithm  (WRGA)  and  its  modification.  In  this  case  an  approximant  Grm 
is  a  convex  linear  combination  of  .  itPm-  We  also  proved  some  convergence 

results  for  WCGA  and  WRGA. 


1.  Introduction 

The  core  problem  of  approximation  continues  to  be  the  development  of  effi¬ 
cient  methods  for  replacing  general  functions  by  simpler  functions.  Some  methods 
were  invented  long  ago  (Fourier  sums,  Taylor  polynomials,  best  approximation  by 
trigonometric  or  algebraic  polynomials  etc.).  More  recently  however,  driven  by 
several  numerical  applications,  the  directions  of  approximation  theory  have  moved 
toward  nonlinear  approximation.  This  includes  the  comparatively  new  subject  of 
nonlinear  to- term  approximation.  It  has  found  applications  in  numerical  solution 
of  integral  equations,  image  compression,  design  of  neural  networks,  and  so  on. 

The  purpose  of  this  paper  is  to  continue  investigations  of  nonlinear  to- term  ap¬ 
proximation.  We  concentrate  here  on  studying  to- term  approximation  with  regard 
to  redundant  dictionaries  in  Banach  spaces.  This  paper  is  based  on  a  combination 
of  ideas  and  methods  developed  for  Banach  spaces  in  [DDGS]  with  the  approach 
used  in  [T]  in  the  case  of  Hilbert  spaces.  The  papers  [DDGS]  and  [T]  contain  de¬ 
tailed  historical  remarks  and  we  refer  the  reader  to  those  papers.  In  this  paper 
we  will  mention  only  those  results  which  are  directly  connected  with  the  results 
presented  here. 

Let  A  be  a  Banach  space  with  norm  1 1  •  1 1 .  We  say  that  a  set  of  elements  (functions) 
V  from  A  is  a  dictionary  if  each  g  G  D  has  norm  one  (||g||  =  1), 

g  G  V  implies  —  g  G  T>, 

1This  research  was  supported  by  the  National  Science  Foundation  Grant  DMS  9970326  and 
by  ONR  Grant  N00014-91-J1343 
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and  span©  =  X . 

We  will  study  in  this  paper  two  types  of  greedy  algorithms  with  regard  to  V. 
For  an  element  /  el  we  denote  by  Ff  a  peack  functional  for  /: 

||F>||  =  1,  Ff(f)  =  ||/||. 

The  existence  of  such  a  functional  is  guaranteed  by  Hahn-Banach  theorem.  Let 

t  :=  be  a  given  sequence  of  positive  numbers  tk  <  1,  k  =  1, _ We  define 

first  the  Weak  Chebyshev  Greedy  Algorithm  (WCGA)  that  is  a  generalization  for 
Banach  spaces  of  Weak  Orthogonal  Greedy  Algorithm  defined  and  studied  in  [T] 
(see  also  [DT]  for  Orthogonal  Greedy  Algorithm). 

Weak  Chebyshev  Greedy  Algorithm  (WCGA).  We  define  ///  :=  /q’t  :=  /. 
Then  for  each  m  >  1  we  inductively  define 
!)•  <-pcm  :=  ^mr  e  ^  is  any  satisfying 

Ff  -  (c pcm )  >  tm  sup  Ffc  ( g ). 

g£V 

2) .  Define 

:=  :=  span{^}™=1, 

and  define  :=  G^T  to  be  the  best  approximant  to  /  from  <Fm. 

3) .  Denote 

/c  . _  fC,T  . _  n 

m  *  Jm  *  J  '-Tra* 

We  define  now  the  generalization  for  Banach  spaces  of  the  Weak  Relaxed  Greedy 
Algorithm  studied  in  [T]  in  the  case  of  Hilbert  space. 

Weak  Relaxed  Greedy  Algorithm  (WRGA).  We  define  /q  :=  /q’t  :=  /  and 
G o  :=  Gq  t  :=  0.  Then  for  each  m  >  1  we  inductively  define 
!)•  ■=  e  X>  is  any  satisfying 

-  G^_x)  >  tm  SupFfL  i(g  -  G^_x). 


2) .  Find  0  <  Am  <  1  such  that 

11/  -  ((1  -  Am)G(n_1  +  =  inf  ||/  -  ((1  -  A )Grm_1  +  A^)|| 

U  \  A  \  1 

and  define 

Grm  ■■=  Gr£  :=  (1  -  \m)Grm  !  +  A m(prm. 

3) .  Denote 

fr  . i?r,r  . j?  /tr 

J  m  *  Jm  *  J  ^  m  * 

The  term  ’’weak”  in  both  definitions  means  that  at  the  step  1).  we  do  not  shoot 
for  the  optimal  element  of  the  dictionary  which  realizes  the  corresponding  sup  but 
are  satisfied  with  weaker  property  than  being  optimal.  The  obvious  reason  for  this 
is  that  we  don’t  know  in  general  that  the  optimal  one  exists.  Another,  practical 
reason  is  that  the  weaker  the  assumption  the  easier  to  satisfy  it  and  therefore  easier 
to  realize  in  practice.  The  Weak  Relaxed  Greedy  Algorithm  provides  incremental 
approximants  discussed  in  [DDGS].  In  [DDGS]  they  also  impose  weaker  assumptions 
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(e-greedy)  on  an  element  of  the  dictionary  than  being  optimal.  For  instance,  for  a 
given  sequence  { en }^L1,  en  >  0,  n  =  1,  2, . . . ,  they  take  0  <  am  <  1  and  gm  £  V 
satisfying 

||  f  (  ( 1  ®-rn)Gm—  1  ~f*  1 1  hlf  |  f  ((1  (Aj Gm __  \  T  OLQ)  1 1  ~t~  Cm 

0<a<l,g£V 

instead  of  trying  to  find  optimal  ones.  Their  approach  is  different  from  ours. 

We  study  in  this  paper  the  questions  of  convergence  and  the  rate  of  convergence 
for  the  two  above  defined  methods  of  approximation.  It  is  clear  that  in  the  case 
of  WRGA  the  assumption  that  /  belongs  to  the  closure  of  convex  hull  of  V  is 
natural.  We  denote  the  closure  of  convex  hull  of  V  by  A\{T>).  It  has  been  proven 
in  [T]  that  in  the  case  of  Hilbert  space  both  algorithms  WCGA  and  WRGA  give 
the  approximation  error  for  the  class  Ai(T>)  of  the  order 

m 

a+E**r1/2- 

k= 1 

We  consider  here  approximation  in  uniformly  smooth  Banach  spaces.  For  a  Banach 
space  X  we  define  the  modulus  of  smoothness 

p{u)  :=  sup  (^-(\\x  +  uy\\  +  \\x-uy\\)-l). 

IMMIj/IHJ  2 

The  uniformly  smooth  Banach  space  is  the  one  with  the  property 

lim  p(u)/u  =  0. 

u— m+ 

It  is  easy  to  see  that  for  any  Banach  space  X  its  modulus  of  smoothness  p(u )  is  a 
convex  function  on  (0,  oo )  satisfying  the  inequalities 

(1.1)  max(0,  u  —  1)  <  p(u)  <  u. 

It  has  been  established  in  [DDGS]  that  the  approximation  error  of  an  algorithm 
analogous  to  our  WRGA  with  =  1,  k  =  1,  2, . . . ,  for  the  class  A\(T>)  can  be 
expressed  in  terms  of  modulus  of  smoothness  of  Banach  space.  Namely,  if  modulus 
of  smoothness  p  of  X  satisfies  the  inequality  p(u)  <  ”fuq,  q  >  1,  then  the  error  is 
of  0(m1/g~1).  We  prove  here  that  both  algorithms  WCGA  and  WRGA  provide 
approximation  for  the  class  A\(T>)  in  a  Banach  space  X  with  modulus  of  smoothness 
p(u )  <  7 uq,  1  <  q  <  oo,  of  order 

m 

d-2)  (i+Y.tir1/p,  p--=Aj- 

k= i  y 

We  prove  also  that  both  algorithms  WCGA  and  WRGA  converge  for  /  £  A\(T>)  if 

,  A  , 

lim  np( - )  =  0, 

n-t  oo  ntn 


for  any  positive  A. 
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It  is  well  known  (see  for  instance  [DDGS],  Lemma  B,l)  that  in  the  case  X  =  Lp, 
1  <  p  <  oo  we  have 


(u)  <  i  UP^P  if  1  -  p  -  2’ 

P  ~  \  (p  —  l)u2 /2  if  2  <  p  <  oo. 

In  Section  4  we  study  a  modification  WRGA(2)  of  WRGA.  The  basic  goal  of 
this  modification  is  to  get  rid  of  solving  the  optimization  problem  at  the  step  2). 
It  is  known  from  [DDGS],  [DT],  [T]  that  in  many  cases  this  can  be  done.  In  our 
modification  WRGA(2)  we  use  at  the  step  2)  the  following  numbers 


Pm  •  — 


2 1 


1 

<3-1 


1  +  tf 


+  ---  +  tp 


m—  1 


m  >  N, 


instead  of  Xm  and  prove  that  this  does  not  effect  the  error  estimate  (1.2). 


2.  Convergence  and  rate  of  approximation  of  WCGA 

We  prove  here  the  following  two  theorems. 

Theorem  2.1.  Let  X  be  a  uniformly  smooth  Banach  space  with  the  modulus  of 
smoothness  p{u).  Assume  that  a  sequence  r  :=  {tk}kLi  satisfies  the  conditions 
1  >  ti  >  t2  >  . . .  and 

,  A  . 

lim  np{ - )  =  0, 

n-t  oo  ntn 

for  any  positive  A.  Then  for  any  f  G  Ai(T>)  we  have 


lim 

ra— ^  oo 


II/, 


C,T\ 

m  I 


0. 


Theorem  2.2.  Let  X  be  a  uniformly  smooth  Banach  space  with  the  modulus  of 
smoothness  p(u )  <  7 uq ,  1  <  q  <  00.  Then  for  a  sequence  r  :=  {tk}kLi,  tk  <  1, 
k  =  1,2,...,  we  have  for  any  f  G  A\(T>)  that 

m 

\\KT\\<c(q,1)(i  +  '£tir1/t',  P~  At, 

k= 1  ^ 


with  a  constant  C(q,ry)  which  may  depend  on  q  and  7. 

We  will  use  the  following  two  simple  and  well-known  lemmas  in  the  proof  of  the 
above  two  theorems. 

Lemma  2.1.  Let  X  be  a  uniformly  smooth  Banach  space  and  L  be  a  finite- dimensional 
subspace  of  X .  For  any  f  G  X  \  L  denote  by  f f  the  best  approximant  of  f  from  L. 
Then  we  have 

Ff-fM  =  0 

for  any  G  L. 

Proof.  Let  us  assume  the  contrary:  there  is  a  f>  G  L  such  that  ||0||  =  1  and 


Ff-fM  =  P  >  °- 
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For  any  A  we  have  from  the  definition  of  p(u)  that 

(2.1)  ||/  -fL-  A0||  +  ||/  -fL  +  A/ll  <  2||/  -  /l||(1  +  tAjjZTM ))' 
Next 

(2.2)  ||/  -  fL  +  A/||  >  Ff_fh (/  -  fL  +  A/)  =  ||/  -  /L||  +  A/3. 
Combining  (2.1)  and  (2.2)  we  get 

(2.3)  |l/-/i-A/||<||/-/i||(l-^  +  2p(^M)). 


Taking  into  account  that  p{u)  =  o(u)  we  find  A'  >  0  such  that 


Then  (2.3)  gives 


(1- 


A'/3 


\\f-h\\ 


+  2  p( 


A' 


11/ -Ml 


))<!• 


11/  ~  II  —  A'/||  <  ||/  -  /l|| 


what  contradicts  the  assumption  that  /  is  the  best  approximant  of  /. 
Lemma  2.2.  For  any  bounded  linear  functional  F  and  any  dictionary  V  we  have 


sup  F(g)  =  sup  F(f). 

g£-D  f£Ai(D) 


Proof.  The  inequality 

sup  F(g)  <  sup  F(f) 

g€V  f€A!(T>) 

is  obvious.  We  prove  the  opposite  inequality.  Take  any  /  G  Ai(T>).  Then  for 
any  e  >  0  there  exist  g\,..-,geN  G  V  and  numbers  a{, ,  aeN  such  that  a\  >  0, 
a\  +  ■  ■  ■  +  aeN  <  1  and 

N 

H/“^a^H  -  C- 

i=  1 

Thus 

N 

F(f)  <  ||F||e  +  F(Va^)  <  e||F||  +  supF(p) 

i=l 

what  proves  Lemma  2.2. 

We  will  also  need  one  more  lemma. 

Lemma  2.3.  Let  X  be  a  uniformly  smooth  Banach  space  with  modulus  of  smooth¬ 
ness  p(u).  Then  for  any  f  G  Ai(T>)  we  have 


wrxw  <  ii rn 


inf(l  -  A tm  +  2p(— 

1 1 J  m— 


-)),  m  =  l,2,... 
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Proof.  We  have  for  any  A 


(2-4)  1 1  frn—  1  “  'Vm-lll  +  Wfm-l  +  -Vm- ill  <  2||/^_1||(1  +  p{ 


A 


l/m—  1 


r)) 


and  by  1)  from  the  definition  of  WCGA  and  Lemma  2.2  we  get 


(2.5) 


A: >  <m s"p  Ud  = 


tm  SUp  Ffo  (0)  >  tmFfc  (/). 

.  _ _ .  Jm  —  1  v  7  J  m  —  1  v  7 

0GAi  (V) 


By  Lemma  2.1  we  obtain 


Ff^Af)  =  l)  =  ll/m-lll- 

Thus  similarly  to  (2.3)  we  get  from  (2.4) 

(2.6)  ||/^||  <  inf  ll/^j  -  ApJ|  <  H/^JI  inf(l  -  Afm  +  2p(Tr-L-)) 

A  A  1 1 J m— 1 1 1 

what  proves  the  lemma. 

Proof  of  Theorem  2.1.  Denote  e{u)  :=  p(u)/u.  Then  we  have 

lim  e(u)  =  0. 

It  is  easy  to  derive  from  the  definition  of  p(u)  that  p(u)  is  a  convex  function.  This 
implies  that  e(u)  is  an  increasing  continuous  function  on  (0,  oo).  The  relation  (1.1) 
implies  that  p( 2)  >  1  or  e(2)  >  1/2.  We  define  inductively  a  sequence  2  >  hi  > 
82  >  •  •  •  >  0.  Assume  hi, ... ,  8m  1  have  been  chosen.  We  choose  8m  <  hm_i  such 
that  the  Am  :=  hm||/)/_1||  satisfies  the  equation  (see  Lemma  2.3) 


what  is  equivalent  to 


Then  (2.6)  implies  that 


^ 


t{8m)  —  ^tm\\fm-l\\- 


(2.7) 

Denote 


ll/mll<ll/m-lll(l-  WU/m-ll 


®m- 1  •  0  8mtm  1 1  fm_  1 1 


Then  using  that  hm+i  <  8m  and  fm+i  <  tm  we  get  from  (2.7) 


A  ®ra—  l(l  Um_i). 
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Thus  by  Lemma  3.4  from  [DT]  we  get 


CLm.—  l  ^ 


2  m 


Sm  < 


mtmll/m-ll 


Assume  the  contrary  that 


Then 


lim  ||/m-ill  =  a  >  0. 


5m  < 


lib  -  , 

mtma 

From  the  definition  of  5m  and  the  monotonicity  of  e(u)  we  get 

^ll/m-rll  =e(5m) 

and 

4  2 

||/^_ill  <  — e(— ^  0  as  to  — >  oo. 

tm  VdljYiOi 

Theorem  2.1  is  proved  now. 

Proof  of  Theorem  2.2.  By  Lemma  2.3  we  have  for  /  E  Ai(T>)  that 

(2-8)  ll/ifll  <  ll/r-ill  inf(l  -  Aim  +  27(-A-iT)'!)- 

A  1 1 J m—  1 1 1 


Choose  Am  from  the  equation 


\\tm  = 

Z  1 1 J m—  1 1 1 


what  implies  that 


Denote 


Am  =  11/^11  —  (47r— wr1. 


Aq  :=  2(47)  9-1 . 


Using  the  notation  p  :=  we  get  from  (2.8) 

ii/ji  <  n/™-iii(i  -  1a  mtm>  =  n/^_1ii(i  -cii/„-irM,)- 

Raising  both  sides  of  this  inequality  to  the  power  p  and  taking  into  account  the 
inequality  xr  <  x  for  r  >  1,  0  <  x  <  1,  we  obtain 

ll/ml|P<ll/m-lHP(l-CII/m-lll7^)- 
By  Lemma  3.1  from  [T]  using  the  estimate  ||/||p  <  1  <  Aq  we  get 

m 

wa?  <Aq[i  +  Y.tpnr1 


what  implies 


\\a\<c(q)(i  +  Y,trny 


Theorem  2.2  is  proved  now. 
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3.  Convergence  and  rate  of  approximation  of  WRGA 


We  prove  here  the  analogs  of  Theorems  2.1  and  2.2  for  the  Weak  Relaxed  Greedy 
Algorithm. 

Theorem  3.1.  Let  X  be  a  uniformly  smooth  Banach  space  with  the  modulus  of 
smoothness  p(u).  Assume  that  a  sequence  r  :=  {ffcjfcTi  satisfies  the  conditions 
1  >  ti  >  t2  >  . . .  and 

,  A  . 

lim  np( - )  =  0, 

n-¥  oo  ntn 

for  any  positive  A.  Then  for  any  f  G  Ai(T>)  we  have 


lim 

ra— ^oo 


11/ 


r,T  | 

m  I 


o. 


Theorem  3.2.  Let  X  be  a  uniformly  smooth  Banach  space  with  the  modulus  of 
smoothness  p(u)  <  ^uq ,  1  <  q  <  oo.  Then  for  a  sequence  r  :=  {tk}kLi,  tk  <  1, 
k  =  1,2, ,  we  have  for  any  f  G  Ai{V)  that 

m 

ll/VII  <  Ci(«,7)(l  +  £*!A1/p,  P  :=  VTT- 

k= 1  5 


with  a  constant  Ci(q,j)  which  may  depend  on  q  and  7. 

Proof  of  Theorems  3.1  and  3.2.  This  proof  is  similar  to  the  proof  of  Theorems  2.1 
and  2.2.  Instead  of  Lemma  2.3  we  use  the  following  lemma  here. 

Lemma  3.1.  Let  X  be  a  uniformly  smooth  Banach  space  with  modulus  of  smooth¬ 
ness  p(u).  Then  for  any  f  G  Ai(T>)  we  have 


WfXW  <  11/ 


r,T 


inf  (1  -  Xtm  +  2p( 

0<A<1 


2A 


11/ 


r,r 


r)),  m  =  1,2,.... 


Proof.  We  have 

//:=/-  ((1  -  A  XmTm)  =  frn—  1  ~  A M  ~  1) 

and 

Similarly  to  (2.4)  we  have  for  any  A 

(3-1)  ll/m.— 1  -  A(Am  -  ^-l)||  +  ||/^-l  +  A(Am  ~  ^-l)||  < 

AllAm-^-rl 


2||//-ill(l  +  p(: 


11/ 


-))• 


m—  1 1 


Next  we  get  for  A  >  0 

ll/m-1  +  A(pL  -  ^_1)||  >  FfL  i(rm_  1  +  A(*4  -  G^_j))  = 
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1 1  frn—  1 1 1  +  ~  Gm  l)  >  ||/m-lll  +  SUP  ~  1)- 

g£V 

Using  Lemma  2.2  we  continue 


=  II/, 


r  I 

m  l  I 


+  Xtm  sup  Ffrn_  ((f)-Grrn_1)>\\fln_1\\+Xtrn\\fln_1\ 

4eAi{T>) 


Using  the  trivial  estimate  \\prm  ~  Gm- ill  <  2  we  obtain  from  (3.1) 

2A 


(3.2) 


Wfin-l  ~H<Prm~G 


r  \ 
m—  1 ) 


<ll/m-lll(l-«m  +  2p(1 


-)), 


ll/m-lll 

what  proves  Lemma  3.1. 

The  remaining  part  of  the  proof  uses  the  inequality  (3.2)  in  the  same  way  as 
the  relation  (2.6)  has  been  used  in  the  proof  of  Theorems  2.1  and  2.2.  The  only 
additional  difficulty  here  is  that  we  are  optimizing  over  0  <  A  <  1.  However,  it  is 
easy  to  check  that  the  corresponding  Xm  chosen  in  the  same  way  as  above  always 
satisfies  the  restriction  0  <  Am  <1.  In  the  proof  of  Theorem  3.1  we  choose  Am 
satisfying  the  equation 


2p(, 


2A, 


)  —  2  ? 


l/m  — 1 

and  in  the  proof  of  Theorem  3.2  from  the  equation 


^Atm  =  27(2A)*||/5 


7*  I 
m—  1 1 


4.  Rate  of  convergence  of  a  modified  WRGA 


We  consider  a  modification  of  WRGA  in  this  section.  This  modification  is  mo¬ 
tivated  by  results  from  [DDGS]  (see  also  [DT])  and  from  [T].  It  was  observed  in 
[DDGS]  that  one  can  replace  Xm  in  the  definition  of  Relaxed  Greedy  Algorithm  by 
1/m  without  loss  of  order  of  approximation  for  Ai(T>).  In  [T]  this  idea  was  used  in 
the  case  of  Weak  Relaxed  Greedy  Algorithm  in  a  Hilbert  space.  We  consider  here 
the  following  modification  of  WRGA  which  we  call  WRGA  of  type  2  and  denote 
WRGA(2).  The  WRGA  defined  in  Section  1  will  be  also  called  WRGA  of  type  1. 
For  a  given  r  =  {Ujfcli  and  1  <  q  <  2  let  N  be  such  that 

1  +  t \  +  ■  ■  ■  +  tpN_1  <2<l  +  tp  +  --  -  +  t PN,  p  :=  —  —  j . 

Then  for  m  <  N  we  define  WRGA(2)  as  WRGA  : 

. _  /ir  nrq  . _  nr 

^ m  *  J  *  J  m ' 

All  notations  for  WRGA(2)  are  those  for  WRGA  with  r  replaced  by  rq.  For  m  >  N 
the  definition  of  WRGA(2)  differs  from  the  definition  of  WRGA  only  at  the  step 
2).  We  define 

G2  ■=  cr  ~  (1 + 


with 


^  rn  *  1  ,p  ’ 

1  '  2^k=  1 

The  assumption  on  N  guarantees  that  0  <Pm 
theorem. 


P  '■= 


Q 

q~  f 


<  1,  m  >  N.  We  prove  the  following 
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Theorem  4.1.  Let  X  be  a  uniformly  smooth  Banach  space  with  the  modulus  of 
smoothness  p(u )  <  7 uq ,  1  <  q  <  2.  Then  for  a  sequence  r  :=  {tk}kLi,  tk  <  1, 
k  =  1,2, ,  we  have  for  any  f  E  Ai{T>)  that 

m 

<  C2(«,7)(l  +  E^r17”’  P  :=  -4y, 

fc=l  ^ 

with  a  constant  C2(q,ry)  which  may  depend  on  q  and  7. 

Proof.  Denote 

n 

T0:=  1.  T„:=(l  + 

fc  =  l 

Then  obviously  Tn  <  2Tn_i  for  all  n.  We  want  to  prove  that 

<  CTm\  m  =  1,..., 

with  a  constant  (7  independent  of  to.  The  proof  goes  by  induction.  For  convenience 
let  us  drop  rq,r  in  the  notations.  Theorem  3.2  gives  the  required  estimate  with 
C  >  Ci(q,  7)  for  to  =  1, . . . ,  N.  We  prove  this  estimate  for  m  >  N.  Assume 

\\fm-i\\  < 

with  some  constant  C\  <  C  which  will  be  specified  later.  Then 

(4.1)  ||/m-l  “  PmfPm  ~  Gm 1)  ||  <  +  2/3m  = 

T~UiCx  +  2 tt'T^i)  <  {Ci  +  2 )Tm\  <  2(Cx  +  2)Tm1. 

Thus  if  C\  satisfies  the  inequality 

(4.2)  2(Ci  +  2  )<C 

then  we  get  the  required  estimate  for  ||/m||  in  this  case.  Assume  now  that 
(4-3)  C\Trnf1  <  H/m-ill  <  CT~f1. 

By  (3.2)  with  p(u)  <  7 uq  we  get 

(4.4)  \\fm\\  <  \\fm-l\\{l-l3rntrn  +  C2f3Ufrn-l\\-q) 
with  C2  :=  21+g7.  Assume  that  C\  satisfies  the  inequality 

(4.5)  2g-1  <  C2lC\. 

Then  using  (4.3)  we  see  that 

fimdm  >  C2ld<fn\\fm-l\\  Q- 


This  implies  that 


Pm  ■—  1  “  fimtm.  +  C2f3fn  \  |  /m_  1 1 1  q  <  1 
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and 

(4-6)  i4n  <  \im. 

Raising  (4.4)  into  the  power  q  and  using  (4.6)  we  obtain 

(4.7)  ||/m||*  <  ||/m-l||9(l-/3m^  +  C'2/3^||/m_1||-^)  =  \\fm-l\\q(l-f3mtm)  +  C2^rn. 
Using  (4.3)  we  get  from  here  that 

\\fm\\qT«m  <  Cq (T.m /T.m_i)q (1  -  2t^nTmp_1)  +  C2 2qt^nTmpq1 . 

Next,  for  1  <  q  <  2  we  have 

(Tm/Tm_,)«  =  (1  +  «,)■''  <  1  +  PmT-r_i 

and 

(Tm/Tm^)q(l  -  2CCi)  <  1  -  tPmTmp i. 

Therefore, 

(4-8)  ll/JI^C  <  C«(l  -  tpmTmp i(l  -  C22«C-«))  <  C* 

if 

(4.9)  Cq  >  C2  2q. 

Now  we  specify  the  constants  C  and  C\  as 

Ci  :=  A^lq 

and 

(4.10)  C  :=  max(4  +  Sy1^,  C\(q,  7)). 

It  is  easy  to  check  that  these  C 1  and  C  satisfy  (4.2),  (4.5),  (4.9)  and  with  the  C 
from  (4.10)  the  inequality  (4.8)  gives 

\\fm\\qTqm<Cq 

what  comletes  the  proof  of  Theorem  4.1. 

Remark.  Theorem  4.1  covers  the  case  1  <  q  <  2.  We  can  modify  the  definition 
of  WRGA(2)  to  extend  Theorem  4.1  to  the  case  2  <  q  <  00.  Denote  in  this  case 
s  :=  q/p  and  choose  N  such  that 

1  +  1 1  +  ■  ■  ■  +  tpN_x  <  2s  <  1  +  tp  +  ■  ■  ■  +  tpN ,  p  :=  ^  — , 

and 


" m  '  1  1  T-^m—  1  ,p  ’ 

1  '  z^fc= 1  U 

Then  using  the  inequalties 

(1  +  x)s  <  1  +  (2s  -  l)x,  0 
(1  +  x)s(l  —  2s x )  <  1  —  x, 

in  the  estimating  the  right  hand  side  of  (4.7) 
some  C. 
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<  X  <  1,  s  >  1, 

0  <  x  <  2-s, 

we  get  the  required  estimate  with 


